Essentials for Modern Data Analysis Systems∗

Earth scientists need to perform complex statistical queries as well as mining queries such as outlier/pattern detection on very large multidimensional datasets produced by AIRS instrument. On top of that, the desired accuracy varies per application, user and/or dataset and it can well be traded-off for faster response time. Towards this end, we have designed and developed a data storage and retrieval system which deploys wavelet transform and provides fast approximate answers with progressively increasing accuracy in support of the scientific queries. We employ a standard web-service infrastructure to assist NASA scientists to interact with AIRS dataset.